System and method of adaptive and progressive descrambling of streaming video

ABSTRACT

A method of distributing digital video sequences in the form of a stream including sequences of data containing digital images, including a modification stage of an original stream by modifying at least a part of the sequences, the modification producing a modified stream of the same nominal format as the original stream, transmitting the modified stream, and a reconstruction stage implemented with a decoder, wherein the reconstruction is adaptive and progressive in function of information coming from a digital profile of the recipient.

RELATED APPLICATION

This is a continuation of International Application No.PCT/FR2003/002915, with an international filing date of Oct. 3, 2003 (WO2004/032478 A2, published Apr. 15, 2004), which is based on FrenchPatent Application No. 02/12271, filed Oct. 3, 2002.

FIELD OF THE INVENTION

This invention relates to the domain of digital video data processing,more particularly to a system for scrambling and recomposing digitalvideo content.

BACKGROUND

WO 01/97520 discloses a device for secure broadcasting, conditionalaccess, controlled visualization, private copy, and storage atrecipients of protected streaming video. That disclosure concerns avideo interface arrangement connecting at least one display device toone video source, in essence consisting, on the one hand of a programmedprocessing unit adapted to manage, more specifically the decoding anddescrambling of any type of streaming video, according to a predefineddescrambling program for the purpose of decoding the video sequence, ofdisplaying it in real time or delayed, storing it, recording it and/orsending it through the telecommunication network, and on the other hand,at least one screen interface, at least one storage unit, one localnetwork interface, one user interface and one control interface, theinterfaces being connected and piloted by a control unit, integrated atthe recipient.

SUMMARY OF THE INVENTION

This invention relates to a method of distributing digital videosequences in the form of a stream including sequences of data containingdigital images, including a modification step of an original stream bymodifying at least a part of the sequences, the modification producing amodified stream of the same nominal format as the original stream,transmitting the modified stream, and a reconstruction step implementedwith a decoder, wherein the reconstruction is adaptive and progressivein function of information coming from a digital profile of therecipient.

This invention also relates to a system for distributing digital videosequences including a server containing means of broadcasting a modifiedstream according to the distribution method, and a plurality of devicesequipped with a descrambling circuit, wherein the server contains inaddition a recorder for the digital profile of the recipient and ananalyzer for the profile of each recipient of the modified stream, therecorder commanding the nature of the complementary informationtransmitted to each of the analyzed recipients.

BRIEF DESCRIPTION OF THE DRAWING

The invention will be better understood with the help of thedescription, given here after strictly for explanatory purposes, of oneway of implementing the invention, with reference to the attachedFIGURE:

the Drawing illustrates a particular way of implementing theclient-server system conforming to the invention.

DETAILED DESCRIPTION

This invention relates to a device capable of transmitting in a securemanner a set of films of high visual quality to a television screenand/or to be recorded on the hard disk of a device connecting thetelecommunication network with the visualization screen such as atelevision or the monitor of a personal computer, while preserving theaudiovisual quality, but avoiding fraudulent use such as the possibilityof making pirated copies of films or of the audiovisual programsrecorded on the hard disk of the decoding device.

The invention also relates to a method for distributing digital videosequences according to a nominal stream format comprising a successionof frames, each containing at least one digital block regrouping acertain number of coefficients corresponding to simple video elementsdigitally coded according to a method defined inside the subject streamand used by all video decoders capable of displaying it to decode itcorrectly. This method comprises:

a preparatory stage comprising modifying at least one of thecoefficients,

a transmission stage

-   -   of a main stream conforming to the nominal format, comprising        planes containing the blocks modified during the preparatory        stage and    -   by a way separate from the main stream, of complementary digital        information allowing to reconstitute the original stream        starting from a calculation, on the destination equipment, in        function of the main stream and the complementary information,        wherein the complementary information is defined as a set of        data (for instance, coefficients describing the original digital        stream or extracted from the original stream) and functions (for        instance, substitution or permutation functions). A function is        defined as containing at least one instruction establishing the        relationship between data and operators. The complementary        information describes the operations to be performed to        recuperate the original stream starting from the modified        stream.

In this invention the term “scrambling” means modification of a digitalstreaming video by appropriate methods so that the stream remains inconformance with the standard or to the standard according to which itwas encoded digitally, while making it suitable to be visualized by areader (or viewer or displayer or player), but altered from the point ofview of human visual perception.

The term “descrambling” means the process of restoring the initialstream by appropriate methods, the restored streaming video afterdescrambling being substantially identical to the initial originalstreaming video. The restoration of the original stream is done on thedestination equipment starting from the modified main stream alreadypresent or sent in real time to the destination equipment and of thecomplementary information sent in real time containing data andfunctions executed by means of digital routines (set of instructions).

The term “profile” means a digital file containing descriptors andspecific user information, for instance, cultural preferences and socialand cultural characteristics, user's habits such as the periodicity ofvideo equipment use, the average duration of visualization of ascrambled film, the visualization frequency of a scrambled sequence, orany other behavioral characteristics related to the exploitation offilms and video sequences. A digital file or a digital table that can beexploited by information system means formalizes this profile.

Many scrambling systems have an immediate effect, whether the initialstream is totally scrambled, or the initial stream is not scrambled atall, and the same applies to the descrambling systems for the videocontent. With this type of rigid systems it is difficult to satisfy therequirements of multi-user, multi-application and multi-servicesclient-server systems, in other words, adapting the services in functionof the different users and their rights.

This invention remedies the inconveniences of the prior art by providingan adaptive and progressive descrambling system for the visualizedcontent in function of the profile and rights of the viewer.

In the invention, an adaptive and progressive descrambling is applied ofthe visualized content in function of the profile and rights of eachviewer. The server sends only the parts of the complementaryinformation, that has a structure characterized by a “granularscalability” to deliver to the viewer a content more or less scrambledin function of certain criteria, profiles and rights. The term“scalability” is defined starting from the English word “scalability”that characterizes an encoder capable of encoding or a decoder capableof decoding an ordered set of binary data in such a manner as to produceor reconstitute a multi-layer sequence. Granularity is defined as thequantity of information susceptible of being transmitted by layer of asystem characterized by any scalability, the system is therefore alsogranular. The granularity is proportional to the degree of scrambling.The streaming video is completely scrambled, one time for all viewers.Next, the server sends all or part of the complementary information insuch a manner that the stream appears more or less scrambled to theviewer. The content of the complementary information sent and thecontent visualized on the visualization screen of the viewer are afunction of each viewer and the server controls and performs thetransmission in real time at the time of the visualization by eachviewer.

In one aspect, the invention relates to a method of distributing digitalvideo sequences in the form of a stream comprising data sequencescontaining digital images, the method comprising a modification stage ofthe original stream by modifying at least a portion of the datasequences, the modification producing a modified stream of the samenominal format as the original stream, the method comprising atransmission stage of the modified stream and a reconstruction stage bymeans of a decoder, characterized by the reconstruction being adaptiveand progressive in function of information coming from a digital profileof the destination viewer.

The modification produces a main modified stream and a complementaryinformation allowing the reconstruction of the original stream by adescrambler, the method comprising a transmission stage of the modifiedstream, and comprising in addition a transmission stage to thedestination equipment of a sub-set of the complementary modificationinformation, the sub-set being determined in function of informationcoming from the digital profile of the recipient.

According to one aspect, the modified main stream is recorded on thedestination equipment prior to the transmission of the complementaryinformation to the destination equipment. According to another aspect,the modified main stream and the complementary information aretransmitted together in real time. Advantageously, determination of thesub-set is performed by a granular scalability method; the quantity ofinformation contained in the sub-set corresponds to a scalability leveldetermined in function of the viewer's profile.

According to yet another aspect, the type of information contained inthe sub-set corresponds with a level of scalability determined infunction of the recipient's profile.

According to a particular aspect of the method, the complementarymodification information contains at least a digital routine capable ofexecuting a function. Preferably, the functions are personalized foreach recipient in function of the connecting session. Advantageously,the complementary information is subdivided in at least two sub-parts.

According to still another aspect, different media can distribute thesub-parts of the complementary information. According to another aspect,the same media can distribute the sub-parts of the complementaryinformation.

According to one method of implementation, the complementary informationis transmitted on a physical vector. According to another aspect, thecomplementary information is transmitted on line.

Advantageously, the digital images are the images I, or P, or B, orplanes S. According to a particular method of implementation, thedigital images use wavelet-based technology.

According to another aspect, the format of the nominal video stream isdefined by the standard MPEG-1 or MPEG-2 or MPEG-4.

Preferably, at least a part of the profile is stored on the destinationequipment.

Advantageously, the type of information contained in the sub-set isupdated based on the behavior of the recipient during the connectionwith the server, or in function of user habits, or in function of datacommunicated by a third party.

According to one aspect, the method comprises a prior stage ofanalog/digital conversion in a structured format, the method beingapplied to an analog video signal.

The invention also relates to a system for distributing video sequencescomprising a server containing means of broadcasting a modified streamconforming to the method described previously and a plurality of devicesequipped with a descrambling circuit, characterized by the fact that theserver contains in addition a means of recording the digital profile ofeach recipient and a means of analyzing the profile of each of therecipients of a modified stream, the means commanding the nature of thecomplementary information transmitted to each of the analyzedrecipients.

According to one aspect, the level (quality, quantity, type) ofcomplementary information is determined for each recipient in functionof the state of its profile at the time of visualization of the mainstream.

A digital streaming video comprises sequences of images, regrouped ingroups of images (“Groups of Pictures”) (GOP's). An image can be of typeI (Intra), P (Predicted), B (Bi-directional). A plane S is a planecontaining a static object, that is a fixed image describing thebackground of the image or a coded plane using a prediction based onglobal motion compensation (GMC) starting from a prior reference plane.The I images are the reference images, they are completely coded and aretherefore of large size and contain no information about the motion. TheP planes are predicted planes starting from prior planes, whether Iand/or P by motion vectors in only one direction, called “forward”. TheB planes are called bi-directional, they are linked to planes I and/or Pthat are preceding or following them by motion vectors in the twotemporal directions (forward and back or “backward”). The motion vectorsrepresent bi-dimensional vectors used for the compensation of themotions, which find the difference of the coordinates between a portionof the current image and a portion of the reference image. The digitalimages can also be obtained with the help of wavelet-based technology(for instance, the fixed images in the MPEG-4 standard), the waveletconcept being an iterative scheme, in other words, the never endingrepetition of the same operation on continuously smaller scale, aprinciple which is well characterized by granular scalability.

The definition of the quantity of information contained in the sub-setis the number of data and/or functions belonging to the complementaryinformation sent to the recipient during the connection.

The type of information contained in the sub-set, corresponds to ascalability level determined in function of the recipient's profile. Thetype is defined as the nature of the data and/or functions belonging tothe complementary information sent to the recipient during theconnection. For instance, the type of data is related to the habits ofthe recipient (hour of connection, duration of the connection,regularity of connection and payments), his environment (lives in alarge city, the current weather) and his characteristics (age, sex,religion, community).

The complementary information is composed at least of functionspersonalized for each recipient in relationship to the connectionsession. A session is defined as starting from the connection time, theduration, the type of the first visualized stream and the connectedelements (recipients, servers).

The complementary information is subdivided in at least two sub-parts,each of the sub-parts can be distributed by different media, or by thesame media. For instance, in the case of distribution of thecomplementary information through several media, a more complexmanagement of the rights of the recipients can be ensured.

Our solution can be applied, as an example and in a non-limiting manner,to streams of the type MPEG-2. A MPEG-2 stream comprises sequences, eachsequence being divided into groups of images, themselves comprisingimages I, P, B. One image containing a certain number of sub-structurescalled “slices”. One slice is a succession of macro blocks. A macroblock is a collection of blocks (4 of luminance, and 2 of chrominance,for instance, for the format 4:2:0), representing a large square 64*64pixels on the decoded image. One block contains the informationcorresponding to a square of 8*8 pixels on the coded image in one of thecomponents of the image (luminance Y, or chrominance Cb or Cr).

During the encoding of a stream in MPEG-2 format, the image (whether areference image or obtained by using a motion compensation) isdecomposed according to its three components (Y, Cb and Cr). Each of thethree sub-images thus obtained is then divided in blocks (graded inslices, macro blocks, images, etc.). These blocks undergo severaltreatments before being inserted in the MPEG-2 stream. First, they gothrough the frequential domain by applying a function to them called DCT(Discrete Cosine Transform) which is a classic algorithm for transposinginformation from the spatial domain to the frequential domain. Followingthis transformation, the coefficients located above left of the newlyobtained block correspond to the weakest frequencies in the twodimensions, while those obtained below right, to the highestfrequencies.

A following treatment comprises quantifying these coefficients so thatthe information corresponding to the very high frequencies, for whichthe human eye is little sensitive, is reduced.

The coefficient located above and to the left of the matrix obtainedafter quantification, called DC coefficient, that corresponds with thecontinuous component of the block. In other words, the average value ofthe block pixels, is then stored in the MPEG-2 stream by optimizing thebinary space it occupies thanks to a differential treatment and avariable length coding. When a DC coefficient is decoded, a first valueis obtained starting from the data present in a block, and then thisvalue is added to a prediction value in order to find the real value.Each time a DC coefficient is decoded, the prediction value is updatedwith the value generated in this way. The other coefficients (63 intotal) called AC are then placed one following the other (in the form ofvectors) in such way that the obtained vector contains the largestpossible number of consecutive null values to apply to it a pair leveltype of coding or “Run level”. This type of coding comprises coding thevector in the form of a series of “run-level” pairs, the “run”corresponding to the number of null values that precede the “level”value in the vector. In this way, the more a vector involves successivenull values, the more efficient this type of coding is.

The invention can, for example, comprise, but in a non limiting way,modifying for a MPEG-2 stream the value of certain fields, notably thecoefficients DC and/or AC and/or of the motion vectors, and this whetherfor one or several blocks, macro blocks, slice, and also whether for oneor all of the colorimetric components Y, Cb, Cr or any of the otherelements of the stream structured to obtain a MPEG-2 stream perfectlyconform to the standard, but with a degraded visual quality, and tostore, in the complementary information, organized in differentscalability layers, the necessary information for a decoder toreconstitute parts of the original stream or the integrality of thestream. For instance, we can modify the values of the coefficients DCand AC of the planes I, the DC of the intra blocks in the planes P andthe vectors of the motions between the planes I, P and B. When theapplication of the server decides not to descramble completely thestream to be displayed for a given viewer or when the rights of a viewerare not sufficient for the server to send to the user the integrality ofthe complementary information, the server can for instance return onlythe real values of the coefficients AC of planes I and the DC of theIntra blocks in the planes P, but not the rest of the modifiedinformation.

The Drawing represents a particular preferred implementation method ofthe client-server system conforming to aspects of the invention.

The original stream (101) can be directly in digital form (111) or inanalog form (11). In the latter case, an encoder (not shown) convertsthe analog stream (11) in digital format (111). Hereinafter, we willindicate the input digital stream as (1). The MPEG-2 stream that we wantto secure (1) passes through an analysis and scrambling system (121)that will generate a main modified stream (122) in MPEG-2 format,identical to the input stream (1) except for certain coefficients havingbeen replaced by values different from the original, and is placed in anoutput buffer storage (122). The complementary information (123), in anyformat, contains information related to the elements of the images thathave been modified, replaced, substituted or displaced, and their valueor location in the original stream.

The stream in MPEG-2 format (122) is then transmitted, via a hertz type,high flow network (4), cable, satellite and the like to the terminal ofthe viewer (8), and more in particular on his hard disk (85). When theviewer (8) makes a request to view the film present on hard disk (85),there are two possibilities: either the viewer (8) does not have all therights necessary to view the film, in this case, the MPEG-2 stream (122)generated by the scrambling system (121) present on the hard disk (85)is passed to the synthesis system (87), via a reading buffer storage(83), which does not modify it and transmits it identically to a classicMPEG-2 display reader (81) and its content, visually degraded by thescrambling system (121), is displayed on the visualization screen (6),

-   -   or the server decides that the viewer (8) possesses the right to        view the film, which can be tested by means of a chip card (82)        based system, linked to the synthesis system (87). In this case,        the synthesis system issues a view request to the server (12)        containing the necessary information (123) for the recuperation        of the original video (101). The server (12) sends then via the        telecommunication networks, of analog or digital telephone line        type, DSL (Digital Subscriber Line) or BLR (Boucle Locale        Radio), via DAB networks (Digital Audio Broadcasting) or via        networks of mobile digital telecommunication (GSM, GPRS, UMTS)        (5) the complementary information allowing the reconstitution of        the video (123) so that the viewer (8) can store it in a buffer        memory (86). The synthesis system (87) proceeds then with the        restoration, in the scrambled MPEG-2 stream that it reads in its        buffer memory (83), the modified fields of which it knows the        positions as well as the original values thanks to the content        of the complementary information read in the descrambling buffer        memory (86) of the video. The quantity of information contained        in the complementary information (123) that is sent to the        descrambling system is specific, adaptive and progressive for        each viewer and depends of his rights, for instance single or        multiple use, the right to make one or more private copies, late        or early payment.

The level (quality, quantity, type) of the complementary information isdetermined in function of each recipient, in function of the status ofthe user profile at the time of the transmission of the main stream andat least a portion of said profile is stored on receiving equipment. Forinstance, in the Drawing, a portion of the viewer profile is recorded onthe chip card (82) linked to the synthesis system (87), as for instancethe connection frequency or the regularity of payments. The rest of theprofile can reside on the server, as for instance the kind of videofilms preferred by the viewer.

Another example is the updating of the recipient's profile, whichdepends also from the connection time with the server (referring tobehavior), namely whether the viewer connects regularly (referring tohis habits) or updating in function of the data recuperated from analready existing consumers database on a server and relative to thisviewer.

Still another example is that the server transmits all the complementaryinformation to the viewer during the first minutes of the visualizationof the film, then, as time passes, transmits less and less complementaryinformation to the viewer so that the main stream is less and lessdescrambled, given in this way to the viewer the effect that his screenbecomes more and more scrambled. This functionality can incite theviewer to purchase the displayed film.

Yet another example is that all or part of the complementary information(123) is transmitted to the viewer on a physical vector such as a memorycard or a chip card (82).

The invention claimed is:
 1. A method of distributing digital videosequences in the form of a stream comprising sequences of datacontaining digital images, comprising: producing a scrambled modifiedstream altered from the point of view of human visual perception fromthe original stream by replacing at least a part of coefficients of thesequences of the original stream by values different from originalvalues, the original stream being in a nominal compressed format otherthan encrypted, said modified stream being of the same nominalcompressed format as the original stream, producing complementaryinformation having a granular scalability structure allowingreconstruction of the original stream by a decoder and comprising theoriginal replaced coefficient and a corresponding location in theoriginal stream and at least one function containing at least oneinstruction, wherein the complementary information describes anoperation to be performed to reconstruct the original stream from themodified stream and the original replaced coefficient in thecomplementary information, transmitting the modified stream to arecipient, transmitting a subset of the complementary information to therecipient, the subset being determined based on a digital profile of therecipient and is adaptive and progressive in function of the digitalprofile provided, and reconstructing at a recipient side of the videosequences from the modified stream and the complementary information byreplacing the modified coefficients from the modified stream by valuescontained in the complementary information at locations indicated by thecomplementary information.
 2. The method according to claim 1, whereinthe main modified stream is recorded on the destination equipment priorto transmission of the complementary information to the destinationequipment.
 3. The method according to claim 1, wherein the modified mainstream and the complementary information are transmitted together inreal time.
 4. The method according to claim 1, wherein the quantity ofinformation contained in the sub-set corresponds to a scalability leveldetermined in function of the profile of the recipient.
 5. The methodaccording to claim 1, wherein the type of information contained in thesub-set corresponds to a scalability level determined in function of theprofile of the recipient.
 6. The method according to claim 1, whereinthe modifying complementary information contains at least one digitalroutine capable of performing a function.
 7. The method according toclaim 1, wherein the complementary information is subdivided in at leasttwo sub-parts.
 8. The method according to claim 1, wherein all or partof the complementary information is transmitted on a physical vector. 9.The method according to claim 1, wherein the complementary informationis transmitted on line.
 10. The method according to claim 1, wherein thedigital images are images I, or P, or B, or planes S.
 11. The methodaccording to claim 1, wherein the digital images use wavelet-basedtechnologies.
 12. The method according to claim 1, wherein at least onepart of the profile is stored on the destination equipment.
 13. Themethod according to claim 1, wherein the type of information containedin the sub-set is updated in function of the behavior of the recipientduring the connection with the server, or in function of user habits orin function of data communicated by a third party.
 14. The methodaccording to claim 1, further comprising a preliminary stage ofanalog/digital conversion in a structured format, the method beingapplied to an analog video signal.
 15. A system for distributing digitalvideo sequences comprising a server containing means of broadcasting amodified stream according to claim 1, and a plurality of devicesequipped with a descrambling circuit, wherein the server contains inaddition a recorder for the digital profile of the recipient and ananalyzer for the profile of each recipient of the modified stream, therecorder commanding a subset of the complementary informationtransmitted to each of the analyzed recipients.
 16. The method accordingto claim 6, wherein the functions transmitted to each recipient arepersonalized for each recipient in function of the session.
 17. Themethod according to claim 6, wherein the sub-parts of the complementaryinformation can be distributed by the same media.
 18. The methodaccording to claim 7, wherein the sub-parts of the complementaryinformation can be distributed by different media.
 19. The systemaccording to claim 15, wherein the subset (quality, quantity, type) ofthe complementary information is determined for each recipient infunction of the status of a profile at the time of the visualization ofthe video sequences.